Image coding and decoding method, corresponding devices and applications

ABSTRACT

The invention concerns a method for coding images using selectively at least two image coding modes, each optimising compression of at least a video sequence image on the basis of different optimisation criteria.

[0001] The technical domain of the invention is that of encoding byreducing the bit rate of animated image sequences, particularly video.

[0002] There are a great number of video encoding applications. Theseinclude (list not exhaustive):

[0003] Digital TV transmission

[0004] Real-time video transmission on several types of networks: IP,mobile, (“Streaming IP”)

[0005] Computer storage of video

[0006] The invention is applicable, in particular, within the frameworkof systems that implement MPEG-type encoding. MPEG-type encoding meansencoding based on temporal prediction and discrete cosine transformationbased on a hard block structure, often with a fixed size, but possiblywith variable size. The two representative standards for this encodingfamily are the MPEG-4 standards of versions 1 to 4 and ITU-T/H.263 up toversion 2. The invention can also be applied within the framework of theCCITT H26L recommendation (for example, see corresponding VCEG-N83d1document).

[0007] The video encoding and decoding diagrams currently offered aredivided into 2 categories:

[0008] Encoding standardised by either ISO/MPEG, or by ITU-T, all basedon the same type of techniques (temporal prediction and discrete cosinetransformation based on a block structure)

[0009] Encoding under development offered by research laboratories thatinvoke a large panel of techniques: wavelet encoding, regional encoding,fractal encoding, meshing encoding, etc.

[0010] Currently, MPEG-4 encoding is considered the trade standard, notonly in standardised encoding, but also in terms of published encoding.

[0011] MPEG-4 or ITU-T/H.263++ type encoding is considered as havingreached its limitations, in particular due to the fixed-size hard blockstructure used as support for all encoding calculations and operations.In particular, the temporal prediction of images within a sequence isnot sufficiently exploited.

[0012] Furthermore, published alternative encoding has not yet achieveda sufficient degree of optimisation.

[0013] Therefore, to allow encoded video sequences at low bit rates tobe obtained, encoders normally reduce the size of images and createtemporal subsamples of the original video sequence. Nevertheless, thesecond technique has the inconvenience of restoring erratic movementsthat are more or less problematic for the user depending on thesubsample level.

[0014] To avoid these erratic movements, the missing images (unencoded)must be regenerated in the decoder using temporal interpolation.

[0015] Nevertheless, current techniques for the temporal interpolationof images do not allow satisfactory results to be obtained, particularlywhen they are implemented in a single decoder. Indeed, these techniquesare the source of visual artefacts related to the movement compensationtechniques based on blocks that only define a single movement vector forall the pixels of a block.

[0016] The purpose of the invention specifically is to remedy thelimitations of previous techniques.

[0017] More precisely, one objective of the invention is to provide atechnique for encoding and decoding image data, which allows a reducedbit rate and/or improved regenerated image quality to be obtained withrespect to known techniques.

[0018] This objective is achieved, according to the invention, with thehelp of an image encoding procedure that selectively implements at leasttwo methods of image encoding, each optimising the compression of atleast one video sequence image depending on the various optimisationcriteria.

[0019] According to several beneficial embodiments, informationregarding the choice of one of the said encoding modes of a decoder maybe known to a decoder according to at least one of the techniques thatbelong to the group comprising the following:

[0020] predefined choice, known at the encoding and decoding level;

[0021] representative information of the choice included in the dataflow comprising of at least some encoded image data;

[0022] representative information of the choice included in the dataflow regardless of the encoded image data;

[0023] determination of choice in predefined manner by the decoder.

[0024] Beneficially, the procedure involves a phase for selecting anencoding mode to apply to said image, which includes at least:

[0025] an initial encoding that optimises the photometric representationof an image.

[0026] a second encoding that optimises a representation of the movementbetween at least two images.

[0027] As such, the present invention involves a new video encodingprocedure by encoding hybridisation, in particular, MPEG-type encodingand encoding using temporal interpolation based on a representation bymeshing, as well as the decoding procedure and the structure of theassociated binary representation.

[0028] Preferably, said second encoding takes into account at least oneprevious image and/or at least one subsequent image encoded using thefirst said encoding.

[0029] Beneficially, said second encoding takes into account a movementvector field calculated from the immediately preceding image encodedusing said initial coding and/or a movement vector field calculated fromthe immediately subsequent image encoded using said initial coding.

[0030] Beneficially, these movement vector fields are applied to themeshing.

[0031] In this case, said vector movement fields can be used todetermine a reduced movement vector field, associated with an imageencoded using said second encoding.

[0032] According to a preferential embodiment of the invention, saidselection phase is based on the implementation of an N fixed factorsubsample, an image on N being encoded using said initial encoding.

[0033] Beneficially, this N value is variable, depending on at least onepredefined criterion.

[0034] According to a specific embodiment, said initial encodingimplements a transformation on image blocks and a temporal prediction byblocks. This transformation is, for example, a DCT-type transformation,Hadamard transformation, wavelet transformation, etc.

[0035] In will be observed that the image blocks are not necessarilysquare, but can take on any form adapted to the needs and meansavailable.

[0036] This initial encoding can, in particular, be MPEG-4 or H26Lencoding.

[0037] In the case of the latter, it is preferable to use type I (intra)images and/or type P (predictable) images (and not, preferably, type Bimages).

[0038] According to another particular aspect of the invention, saidsecond encoding is based beneficially on the implementation of anM-level hierarchical meshing, where M is greater than or equal to 1,and, for example, triangular meshing.

[0039] In this case, the procedure preferably involves a managementphase of the occlusion areas, or blanking areas.

[0040] The data produced can be grouped into a single flow.Beneficially, at least two data flows that can be transmitted onindependent transmission paths can be expected.

[0041] Beneficially, said data flows belong to the group that comprises:

[0042] a global header;

[0043] image data encoded according to said initial encoding;

[0044] image data encoded according to said second encoding.

[0045] Flow transmission can therefore take place independently. Inparticular, this allows progressive and/or partial decoding of images,depending on means and needs.

[0046] Therefore, according to a particular embodiment of the invention,the following aspects are exploited:

[0047] Optimisation driven by constitutive modules of standardised MPEGor ITU-T/H.263-type encoding

[0048] Powerful temporal prediction and related error encoding formeshing-based techniques.

[0049] Indeed, the meshing-based approach helps avoid the effects ofnormal blocks thanks to the use of continuous movement fields.Furthermore, the meshing technique allows blanking “objects” to bedetected, as well as error encoding that is properly adapted to theseareas. In addition, by combining MPEG-type error encoding around theseareas, the efficiency of interpolation can be substantially improved ata cost much lower than that of the bi-directional images (type B images)offered by MPEG.

[0050] Therefore, basic information can be efficiently encoded with lowtemporal resolution using MPEG-type encoding, with good quality, and thefull flow of the sequence can then be restored using encoding ininterpolated mode by meshing.

[0051] The invention also involves, of course:

[0052] procedures for decoding an image signal encoded using theencoding procedure described above;

[0053] procedures for encoding an image signal encoded using theencoding procedure described above;

[0054] devices for decoding an image signal encoded using the encodingprocedure described above (including, beneficially, means fordetermining at least part of a vector field and/or at least a part ofthe blanking areas, similar to those implemented during encoding;

[0055] devices for storing at least one image signal encoded using theencoding procedure described above

[0056] encoding, transmission and/or decoding systems of an image signalencoded using the encoding procedure described above (the choice of oneof the said encoding modes may be known beneficially to a decoderaccording to at least one of the techniques that belong to the group,comprising the following:

[0057] predefined choice, known during encoding and decoding;

[0058] representative information of the choice included in a data flowcomprising at least some of the encoded image data;

[0059] representative information of the choice included in a data flowregardless of the encoded image data;

[0060] determination of the choice in an intrinsic manner, by thedecoder;

[0061] the computer program products for encoding and/or decoding of animage signal encoded using the encoding procedure;

[0062] data support for such a program.

[0063] The invention also involves image data signals comprising encodeddata according to the procedure described above.

[0064] Beneficially, this signal comprises at least one indicator thatspecifies whether the procedure has been activated or not.

[0065] Preferably, the signal comprises data that specifies the framestructure, at the beginning of the video sequence and/or at each signalframe.

[0066] Beneficially, a sequence that is encoded using said secondencoding begins with a header that indicates the number of framesencoded according to this second encoding.

[0067] According to a specific embodiment, the signal comprises at leasttwo data flows, which can be transmitted on separate transmission paths.

[0068] In this case, said data flows belong beneficially to the groupcomprising:

[0069] a global header;

[0070] image data encoded according to said initial encoding;

[0071] image data encoded according to said second encoding.

[0072] The invention finds applications in various domains and, inparticular, in domains belonging to the group comprising:

[0073] digital TV;

[0074] real-time video over IP network;

[0075] real-time video over network to mobiles;

[0076] image data storage.

[0077] Other features and advantages of the invention will be moreevident on reading the description of a preferential embodiment of theinvention, provided as a simple illustrated, non-limiting example, aswell as the drawings attached in which:

[0078]FIG. 1 is a schematic diagram of the encoding of the invention;

[0079]FIG. 2 illustrates an example of the hierarchical meshingstructure for the movement;

[0080]FIG. 3 illustrates the affine interpolation principle on atriangular mesh;

[0081]FIG. 4 is an example of blanking detected by covering triangles;

[0082]FIG. 5 illustrates the transformation process of any imagetriangle in a symmetrical square matrix;

[0083]FIG. 6 illustrates the transformation of any triangle into arectangle isosceles triangle;

[0084]FIG. 7 illustrates a hierarchical meshing and representation bythe related quaternary tree;

[0085]FIG. 8 is an example of an encoding decision for the hierarchicalmeshing;

[0086]FIG. 9 illustrates the global structure of a binary streamaccording to the invention;

[0087]FIG. 10 illustrates a block diagram of a decoder according to theinvention.

[0088] The embodiment of the invention described below essentiallyconsists in the hybridisation of MPEG-type encoding, for example MPEG-4,using meshing encoding that operates in interpolated mode, also called Bmode or B images in the MPEG standards.

[0089] It should be noted that the MPEG-4 encoding mentioned here can bereplaced by any encoder based on equivalent techniques, that is, using atemporal prediction and discrete cosine transformation based on a blockstructure, and quantifications and entropic encoding for the generatedinformation. In particular, ITU-T/H.263++ encoding can be substitutedfor MPEG-4 encoding.

[0090] For each image of the sequence coming into the encoder, theencoder decides according to a specific decision-making process (forexample, fixed factor temporal subsampling) of the encoder with theMPEG-4 encoding module, or with the meshing-based encoding module.

[0091] The images encoded in meshing mode use, as a reference for theirtemporal prediction, images encoded in MPEG-4 mode located immediatelybefore or immediately after the group of images encoded in meshing modeto which they belong.

[0092] The key point of the compression efficiency of the invention isthat the meshing-based movement compensation leads to a very powerfultemporal prediction, with a very low associated encoding cost.

[0093] Indeed, this technique:

[0094] Takes into account various types of movement within the images.

[0095] Specifically processes the covering and uncovering of areas dueto the movement of objects.

[0096]FIG. 1 provides an overview of the principle of the encoder.

[0097] First, incoming images are directed to either the MPEG encodingmodule or the meshing-based encoding module, according to a givendecision method, for example, according to a predefined rhythm: 1 imagein N is encoded in MPEG, the rest are encoded in meshing interpolatedmode. It is noted that N_(k) represents the number of images encoded inMPEG mode.

[0098] All the other I_(k)N_(k)<l<N_(k+1) images are encoded by ameshing based encoder, for example triangular, that operates ininterpolated mode, called B mode. The general principal of this encoderis as follows:

[0099] 1. Calculation of before and after movement fields between N_(k)and N_(k+1) images. These fields are modelled in the form of triangularmeshings.

[0100] 2a. Estimation of predictable areas: during interpolation, themovement between It and It′ is estimated. If the movement vectorapplication does not lead to a conversion (which means that one is inthe presence of a predictable area), movement compensation is performedby weighting the vectors using k(0≦k≦1) scalar in order to interpolateI_(t+k) with t+k belonging to ]t, t′[. There are three possibleapproaches to movement compensation, as described below.

[0101] 2b. Estimation of non-predictable areas: detection ofnon-predictable blanking areas, in Il images to be encoded, from theawareness of these movement fields.

[0102] 3. Specific encoding of these blanking areas according to one ofthe following three possible modes:

[0103] Prediction with one of the reference images (N_(k), N_(k+1), orthese images compensated during movement with their movement fields)without movement compensation, then encoding the prediction error with atriangular meshing-based technique.

[0104] Prediction with one of the reference images (N_(k), N_(k+1), orthese images compensated during movement with their movement fields)with intra-image movement compensation, then encoding the predictionerror with a triangular meshing-based technique.

[0105] Intra-image encoding based on a triangular meshing-basedtechnique

[0106] 4. Optionally, mode P MPEG-type encoding of the residualprediction error or encoding, limited to an area around the blankingarea.

[0107] As mentioned above, movement compensation can be performed usingthree approaches: with an estimate before, with an estimate after orwith an estimate before and an estimate after.

[0108] 1) with an estimate before:

[0109] During interpolation, the movement between I_(t1) and I_(t2) isestimated. If the movement vector application does not lead to aconversion (which corresponds to the predictable areas), then movementcompensation is performed by weighting the vectors with k=m/(t1+t2)(0<k<1) scalar in order to interpolate Im with m belonging to [t1, t2].The El estimated image is obtained.

[0110] 2) with an estimate after:

[0111] During interpolation, the movement between I_(t2) and I_(t1) isestimated. If the movement vector application does not lead to aconversion (which corresponds to the predictable areas), then movementcompensation is performed by weighting the vectors with k′=1-m/(t1+t2)(0<=k′<=1) scalar in order to interpolate Im with m belonging to [t1,t2]. The E2 estimated image is obtained.

[0112] 3) both together:

[0113] During interpolation, the movement between I_(t1) and I_(t2) andbetween I_(t2) and I_(t1) is estimated. If the movement vectorapplication does not lead to a conversion (which corresponds to thepredictable areas), then movement compensation is performed by weightingthe “before” vectors with (0<=k<=1) scalar, and E1 is obtained. The sameis done with the “after” vectors using the k′ (0<=k′<=1) scalar, and E2is obtained. The estimated image is then E=aE1+(1-a)E2 with (0<=a<=1).

[0114] Every 2 bits, the best solution is taken after calculating thePSNR between the estimated solution and the related source image.

[0115] 1. Calculating Before and After Movement Fields Between N_(k) andN_(k+1) Images

[0116] The before and after movement fields between N_(k) and N_(k+1)images are calculated, in the form of hierarchical meshing, for exampletriangular, T^(b) _(k) and T^(b) _(k+1) as indicated in FIG. 2.

[0117] These meshings are obtained by dividing certain meshes, forexample, triangular meshes are divided into 4 sub-triangles, accordingto a specific criterion during the movement estimation process. At eachhierarchical level, division decisions are taken or not for each mesh.Once these divisions have been decided upon, the adjacent meshes of thedivided meshes are divided in order to maintain a uniform meshingstructure. The initial meshing, before division (top of the hierarchy)can take any form.

[0118] In the example in FIG. 2, the movement estimator decides todivide triangles 3 and 8. This leads to the division of triangles 2, 4,7, and 9. The process is repeated up to a predefined hierarchy level.

[0119] In the case of triangular meshings, the movement field expressiondefined by the triangular meshing T is given for each triangle e by:$\begin{matrix}{{\overset{->}{d}\left( {\overset{->}{p},D} \right)} = {\sum\limits_{\{{{ver}{(e)}}\}}{{\psi_{n}^{e}\left( \overset{->}{p} \right)}{\overset{->}{d}}_{n}}}} & \quad & {\forall{{\overset{->}{p}\left( {x,y} \right)} \in e}}\end{matrix}$

[0120] where:

[0121] e indicates the triangular element of T containing the currentpoint {right arrow over (p)} of the x and y coordinates

[0122] {ver(e)} indicates all three nodes or peaks, numbered i, j, k ofpositions {right arrow over (p)}_(i), {right arrow over (p)}_(f), and{right arrow over (p)}_(k)

[0123] Ψ₁(l=i,j,k) represents the barycentrical coordinates of point{right arrow over (p)} (x,y) in the e_(i,j,k) triangular element with:$\left\{ \begin{matrix}\left\{ {{\sum\limits_{{l = i},j,k}^{{{\Psi_{l}^{e}{({x,y})}} = {\alpha_{l}^{e} + {\beta_{l}^{e}x} + {\gamma_{l}^{e}y}}},\alpha_{l}^{e},\beta_{l}^{e},{\gamma_{l}^{e} \in \Re}}{\Psi_{l}^{e}\left( {x,y} \right)}} = 1} \right. & {{{if}\quad {\overset{->}{p}\left( {x,y} \right)}} \in e_{i,j,k}} \\{{\Psi_{l}^{e}\left( {x,y} \right)} = 0} & {otherwise}\end{matrix}\quad \right.$

[0124] This type of model defines a fully continuous field. Furthermore,it allows full control of representation accuracy, an essential featurefor compression.

[0125] At each level of the meshing hierarchy, nodal movement vectorsare calculated in order to minimise prediction errors. Variousmeshing-based movement estimators can be used, for example thosedescribed in patent FR No. 98 11227, or FR No. 99 15568.

[0126] The important point to remember is that the final meshing is theresult of a hierarchical process from the initial meshing by divisions.This hierarchical nature is indeed used to advantage for thedifferential encoding of nodal movement vectors between a node and itsparent nodes (the edges of the arc in which it is inserted). The meshingstructure is recalculated in the decoder using the knowledge from theinitial meshing, as well as the mesh division indicators.

[0127] Therefore, at the end of the process, 2 movement meshings areobtained for each group of images comprised of N_(k) and N_(k+1) imagesused to regenerate all the images of the group.

[0128] 2. Detection of Blanking Areas

[0129] From these 2 meshings, the blanking areas occlusions, that is thenon-predictable areas in the N_(k) image obtained from the N_(k+1) imageor vice versa, from the covering or uncovering of objects, are detected.

[0130] These areas are simply defined by the covering of triangles afterthey have been moved by their nodal vectors.

[0131] The figure illustrates the basic blanking detection on theuncovering of triangles after movement.

[0132] The encoder can continue the movement estimate by deactivatingthe triangles in the blanking areas, in order to obtain movement vectorsthat are less biased.

[0133] Nevertheless, this is strictly an internal feature of theencoder's strategy, and at the end, it is the two T^(b) _(k) and T^(f)_(k+1) complete movement meshings that are encoded and inserted in thebinary stream. The decoder is then able to find the blanking areas usingthese 2 meshings.

[0134] These blanking areas are defined in the N_(k) and N_(k+1) imagesand once they are detected, their corresponding triangles are labelledas a result, both in the encoder and in the decoder.

[0135] Nonetheless, the encoder must know where these areas are found inthe N_(k)+1 and N_(k+1)−1 images. These images are simply obtained byprojecting the T^(b) _(k) and T^(f) _(k+1) meshings on the image to beencoded by applying nodal movement vectors that are renormalized to takeinto account the temporal distance between the current image and thereference N_(k) or N_(k+1) image.

[0136] 3. Blanking Area Encoding:

[0137] For each blanking area, the reference image for a possibleprediction is selected among I_(Nk), I_(N(k+1)), as well as I^(c) _(l),which is the image obtained at l, by movement compensation with theT^(b) _(k) or T^(f) _(k+1) meshing, at a level in which there is as yetno mesh covering.

[0138] More precisely, the choice between I_(Nk) and I_(N(k+1)) simplydepends on the T^(b) _(k) or T^(f) _(k+1) meshing that generated thecurrent blanking area. Then, this meshing is used to predict the imageand provide I^(c) _(l). The choice between I^(c) _(l) and I_(Nk) orI_(N(k+1)) is based on a prediction error criterion: the image causingthe weakest error is maintained. Therefore, it is simply a matter ofinserting 1 bit in the binary flow, per area, to encode the choice ofthe prediction that was maintained.

[0139] Take not of I, the selected reference image.

[0140] The rest of the encoding of these areas includes 2 stages:

[0141] A prediction stage.

[0142] An encoding stage of the prediction error or of the originaltexture in the case of an incorrect prediction.

[0143] 3.1 Residual Prediction of the Blanking Area Texture

[0144] Three methods can be used, in an exclusive manner. The decisionis based on the lowest error criterion.

[0145] Method 1:

[0146] The values Y, U and V of a pixel in the area are simply those ofthe pixel located in the same reference image I₁. That is, Î₁ theresulting image. The prediction error between Î and Il is then encoded.

[0147] Method 2:

[0148] A movement estimate is therefore performed between I₁ (the imageto be encoded) and Î₁ (the result of the prediction from method 1) onthe blanking area. The resulting meshing, that stems from the lastmeshing level T_(r), r=k or k+1, before the meshes are covered, istherefore encoded as are its nodal movements. Lastly, the residualprediction error is encoded according to a procedure defined later.

[0149] Method 3:

[0150] No prediction is performed and the original values of the area'spixels are encoded.

[0151] 4. Encoding the Texture or Prediction Error of Blanking areas.

[0152] The original texture and prediction error undergo the sameencoding, for which the principle is as follows:

[0153] It is assumed that an initial triangular meshing was able to bedefined from the T_(r) meshing movement maintained for the prediction ofthe area to be encoded. The way of obtaining this initial meshing willbe described later.

[0154] The texture is therefore approximated on each mesh according to achoice:

[0155] Meshes with many high frequencies are encoded based on thetransformation by discrete cosine called DCT.

[0156] The smoother meshes are encoded using a refined model of finishedelements.

[0157] Again, a hierarchical approach will be used to reduce the cost ofencoding the representation through meshing.

[0158] The approach used allows the low cost of encoding related to aregular hierarchy of meshes to be maintained whilst permitting localadaptation of image contents that the irregular decomposition of meshesallows.

[0159] From the thick initial meshing of the area, meshes are subdividedinto 4 triangular sub-meshes up to a given level.

[0160] On the last level, an optional permutation of the quadrilateraldiagonals generated by 2 adjacent triangles can be implemented, if thisresults in a reduction of the approximation error.

[0161] 4.1 Initialisation of Texture Meshing on Blanking Areas

[0162] This meshing is simply given by the last T_(r) level (meshingresulting from the movement of T_(k) or T_(k+1) depending on thedirection maintained) before the conversions appear in the area underconsideration. Therefore, a texture meshing that is inserted naturallyinto the movement meshing is obtained, because it is extracted from thelatter.

[0163] 4.2 Representations Used for the Texture on Triangles

[0164] Two representations are combined: affine interpolation andtriangular 4 DCT

[0165] Affine Interpolation

[0166] The triangular meshing nodes contain the photometric (colour,error) information and interpolation for the points inside the triangleis performed by a Lagrange finished element, also called affineinterpolation.

[0167] The v({right arrow over (p)}) value of point {right arrow over(p)} (x,y) inside the triangle e_(i,j,k) defined by the 3 nodes {rightarrow over (p)}_(l),l=i,j,k is provided by the following equation:$\left. {{v\left( \overset{->}{p} \right)} = {\sum\limits_{{l = i},j,k}{{\psi_{l}^{e}\left( {x,y} \right)}{\overset{->}{p}}_{l}}}} \right)$

[0168] where Ψ_(l)(l=i,j,k) represents the barycentrical coordinates ofthe point.

[0169] v({right arrow over (p)}) can be one of the Y, U, or Vphotometrical components of the point or the prediction error for thesecomponents.

[0170] Several methods can be used to calculate the nodal values, inparticular, the method of lesser squares.

[0171] Discrete Cosine Transformation (DCT) on Triangles

[0172] The principle of the method consists in transforming any triangleinto a rectangular isosceles reference. The content of this triangle istherefore rendered symmetrical with respect to the hypotenuse to providea symmetrical square matrix (FIG. 4).

[0173] A classic DCT (square) is therefore applied to this matrix. Itcan be shown that the transformed matrix is also symmetrical. Only thecoefficients of its lower triangle are therefore quantified thenstatistically encoded (entropic encoding).

[0174]FIG. 4 describes the various stages of the process: selection ofthe T triangle, affine transformation of the T triangle into a T′rectangle isosceles triangle. Through affine transformation, thetriangle pixels are no longer located on a regular orthogonal graph, andresampling of the photometric values inside the reference triangle isrecommended. To do so, a process similar to the process for movementcompensation in the image (during affine transformation) is used, usingan interpolater, for example bilinear.

[0175] The affine transformation F and its inverse F¹ are defined by thefollowing equations: $\begin{matrix}{F\text{:}\quad \left\{ \begin{matrix}{X = {\frac{{\left( {x_{2} - x_{1}} \right)\left( {y_{1 -}y} \right)} + {\left( {y_{1} - y_{2}} \right)\left( {x_{1} - x} \right)}}{\left( {x_{3} - x_{1}} \right) + \left( {y_{2 -}y_{1}} \right) + {\left( {y_{1} - y_{3}} \right)\left( {x_{2} - x_{1}} \right)}}N}} \\{Y = {\frac{\left( {x_{1} - x_{3}} \right) + \left( {y_{1} - y} \right) + {\left( {y_{3} - y_{1}} \right)\left( {x_{1} - x} \right)}}{{\left( {x_{3} - x_{1}} \right)\left( {y_{2} - y_{1}} \right)} + {\left( {y_{1} - y_{3}} \right)\left( {x_{2} - x_{1}} \right)}}N}}\end{matrix} \right.} \\{F^{- 1}\text{:}\quad \left\{ \begin{matrix}{x = {x_{1} + {\left( {x_{3} - x_{1}} \right)\frac{X}{N}} + {\left( {x_{2} - x_{1}} \right)\frac{Y}{N}}}} \\{y = {y_{1} + {\left( {y_{3} - y_{1}} \right)\frac{X}{N}} + {\left( {y_{2} - y_{1}} \right)\frac{Y}{N}}}}\end{matrix} \right.}\end{matrix}$

[0176] The photometric values M(i,j) of the T′ triangle (therefore ofthe M block rendered symmetrical of T′) are obtained by the F⁻¹ inversetransformation, and then the Γ interpolation of the texture of the imageto encode:

M(i _(y) ,jx)=I _(Γ)(P _((x,y)))=I _(Γ)(F ⁻¹(Q_((x,y))))

[0177] where:

[0178] the M(i_(Y),j_(X)) coefficient is the value at the Q(X,Y) pointwhere the transformed P(x,y) is F⁻¹(Q)

[0179] I_(Γ) indicates the interpolater used to calculate the imagevalue at the point, the coordinates that are potentially non-integers.

[0180] The {circumflex over (Γ)} regeneration of the Γ texture is givenby:

{circumflex over (Γ)}(P _((x,y)))=I _({circumflex over (Γ)})(Q_((x,y)))=I _({circumflex over (Γ)})(F(P _((x,y)))

[0181] where I_(Γ) indicates the interpolated texture from the M′ blockvalues, a quantified version of M.

[0182] This technique can only be applied to non-null area triangles.But, by definition, this type of triangle does not need textureencoding.

[0183] Contrary to SADCT (DCT adapted to a form), this transformationdoes not guarantee the perfect regeneration after inversetransformation, even if there is no quantification.

[0184] In order to reduce regeneration errors, a scale factor α isintroduced to calculate the M_(i) block (size N_(i)×N_(i)) for trianglei:

N _(i) =E(α{square root}{square root over (2A _(i)))}

[0185] where:

[0186] E is the complete part by excess,

[0187] A_(i) is the triangle i area.

[0188] In fact, α=1 provides for an interesting compromise, which ismore effective for triangles that are close to an isosceles. The caseα<1 is used in conjunction with the quantification step to compress theinformation volume.

[0189] Once the Mi block is defined for each triangle, a classic DCTtransformation is applied to them, and the transformed coefficients arequantified according to several possible procedures, for example,uniform scalar quantification, or even incremental quantification withthe frequency of the transformed coefficient. The use of well known MPEGor JPEG quantification matrices is also possible.

[0190] The DCT expression is given by: $\begin{matrix}{{F\left( {u,v} \right)} = {\frac{1}{\sqrt{NM}}{\sum\limits_{i = 0}^{N - 1}{\sum\limits_{j = 0}^{M - 1}{{\lambda (u)}{\lambda (v)}\cos \frac{\left. {{2i} + 1} \right)u\quad \pi}{2N}\cos \frac{\left( {{2j} + 1} \right)v\quad \pi}{2M}{f\left( {i,j} \right)}}}}}} \\{{\lambda (\xi)} = \left\{ \begin{matrix}\frac{1}{\sqrt{2}} & {{{if}\quad \xi} = 0} \\1 & {otherwise}\end{matrix} \right.}\end{matrix}$

[0191] The F(u,v)=F(v,u) relationship is present because:

[0192] f(i,j)=f(j,i) (∀u,v,i,j=0, . . . ,N−1) by definition.

[0193] As a result, it is enough to calculate only the coefficients ofthe lower part of the transformed matrix.

[0194] 4.3 Global Encoding of Texture

[0195] As previously indicated, a uniform hierarchical meshing is usedwhich is obtained by dividing each triangle of a given level of thehierarchy into 4 sub-triangles, and inserting nodes in the middle of thearcs. The process is repeated until the maximum level is reached. Thishierarchy of triangles is also represented and managed by the encoder inthe form of a quaternary tree (FIG. 5). Note that only the trianglesincluded in the area to encode are taken into account. The constructionprocedure of the basic initial meshing guarantees that all the trianglesof the meshing hierarchy belong to the area to encode.

[0196] The encoding process by meshing a blanking area can be summarisedas follows:

[0197] 1. An interleaved hierarchical meshing is defined on the area toencode by creating a regular initial meshing and then repeatedlysubdividing triangles into 4 sub-triangles and inserting new nodes inthe middle of the arcs. The node values are calculated in order tominimise the approximation error of the area due to meshing.

[0198] 2. The pixel values are moved closer together using affineinterpolation on the triangle that contains the pixels using the nodevalues.

[0199] For each triangle in the hierarchy, the approximation error E isevaluated and then the various representation and encoding methods aredecided upon based on 2 thresholds: σ₁ and σ₂:

[0200] 1. if E<σ₁, affine interpolation on the triangle is sufficient.

[0201] 2. If σ₁<E<σ₂, the thinnest decomposition of the triangle must beused to obtain a good approximation, always using affine interpolation.

[0202] 3. if E>σ₂, the triangle is textured and the affine interpolationerror is encoded using DCT.

[0203] Lastly, on the thinnest meshing, the reduction of the errorproduced by the permutation of the quadrilateral diagonals formed by 2adjacent triangles is tested. If a positive result is obtained, thispermutation is validated.

[0204] According to the encoding methods chosen for the differenttriangles, the various data are encoded in the following manner.

[0205] The YUV nodal values are first predicted using the values of theparent nodes (edges of the arc where the current node is inserted). Thedifference between the node value and its predicted value is thereforequantified.

[0206] Lastly, the quaternary tree structure (including the division andnon-division indicators of the triangles), the diagonal permutationindicators, the differential nodal values of YUV and the quantified DCTcoefficients are encoded using an arithmetic encoder and inserted in thebinary stream.

[0207] 5. Summary of Encoded Information in the Binary Flow of FramesEncoded by Meshing

[0208] Each group of frames encoded in meshing mode between N_(k+1) andN_(k+1)−1 (where N_(k) N_(k+1) are the preceding and subsequent frames,respectively, encoded in MPEG mode) is represented as a whole in thebinary flow.

[0209] The driven data comprises, in encoded form:

[0210] A header for the set of frame groups, including, among others,the real number of encoded frames.

[0211] The movement meshings (structure and movement vectors of nodes)T^(b) _(k) and T^(f) _(k+1.)

[0212] The prediction or original error texture, for each image of thegroup.

[0213] 6. Global Structure of the Binary Stream

[0214] The global binary stream comprises a succession of frames encodedin MPEG mode and groups of frames encoded in meshing mode, as indicatedin FIG. 8.

[0215] The global header of the binary stream that represents theencoded sequence contains, among others, the indication of the hybridencoding.

[0216] The part of the binary stream that corresponds to a group offrames encoded in meshing mode begins with a header that indicates,among other things, the number of frames actually encoded, and may benull.

[0217] The various data flows (binary streams) that correspond,respectively, to the global header of the encoded sequence, the imagesencoded in MPEG and the groups of i images encoded in interpolatedmeshing mode can be sent through different, independent paths ifrequired. In particular, the encoding procedure allows hierarchical (orscalable) encoding of the sequence, that is, decoding that only uses apart of the total bit rate.

[0218] 7. Decoding Procedure

[0219]FIG. 9 provides an overview of the decoding principle.

[0220] Firstly, the header decoding allows hybrid decoding to beactivated.

[0221] Then, the decoder recognises for each part of the binary streamthat corresponds to an independent entity if it is a frame encoded inMPEG-4 or a group of frames encoded by meshing.

[0222] MPEG-4 frames are provided to the MPEG-4 decoding module, and thegroups of frames encoded in meshing mode are provided to the decodingmodule through meshing.

[0223] 7.1 Meshing-Based Decoding

[0224] Firstly, the T^(b) _(k) and T^(f) _((k+1)) movement meshings forthe I₁, N_(k)<1<N_(k+1) image group are decoded.

[0225] Next, the blanking areas for these images are found according tothe same procedure as in the encoder.

[0226] Then, the pixels outside the blanking areas are simplyinterpolated using the I_(Nk) and I_(Nk+1) images and the T^(b) _(k) andT^(f) _(k+1) movement fields.

[0227] Meshing of the thickest texture (top of the hierarchy) is foundfor each blanking area according to a procedure that is identical tothat of the encoder procedure.

[0228] The information related to the corresponding hierarchical meshing(triangle division indicator, affine interpolation decisions or DCTencoding, differential nodal YUV values and quantified DCT coefficients)is therefore decoded and the pixel YUV values of these areas areregenerated.

1. Image coding process, characterised by the following steps: selectingtwo images IT₁ and IT₂, split by series of at least one image that hasto be undersampled; coding said IT₁ and IT₂ images delimiting saidseries of images through a first mode of image coding, optimizingsignificantly a photometric representation of an image; coding eachimage of the series of images through a second mode of image codingoptimizing a representation of movement between at least two images,based on a determination of at least one field of movement vectorsdescribed by a mesh made up of nodes to which is affected a movementvector, so that each image is coded according to only one coding mode.2. Encoding method as claimed in claim 1, wherein a piece of informationrepresenting a choice of one of said encoding modes is known by adecoder according to at least one of the techniques belonging to thegroup comprising: a predefined choice, known at the encoding anddecoding levels; a piece of information representing said choice andincluded in a data flow comprising at least some encoded image data; apiece of information representing said choice and included in a dataflow regardless encoded image data; determining said choice in anintrinsic manner, by the decoder.
 3. Encoding method as claimed ineither of claims 1 and 2, wherein it comprises a step of selecting anencoding mode to be applied to said image, among which at least: aninitial encoding substantially optimising a photometric representationof an image. a second encoding substantially optimising a representationof movement between at least two images.
 4. Encoding method as claimedin claim 3, wherein said second encoding takes into account at least oneprevious image and/or at least one subsequent image encoded using saidinitial encoding.
 5. Encoding method as claimed in claim 4, wherein saidsecond encoding takes into account a movement vector field calculatedfrom an immediately preceding image encoded using said initial codingand/or a movement vector field calculated from an immediately subsequentimage encoded using said initial coding.
 6. Encoding method as claimedin claim 5, wherein said movement vector field is applied to a meshing.7. Encoding method as claimed in either of claims 5 and 6, wherein saidmovement vector fields are used to determine a deduced movement vectorfield, related to an image encoded using said second encoding. 8.Encoding method as claimed in any of claims 5 to 7, wherein said secondencoding implements an estimate of movement before, between an I_(t1)image and an image after I_(t2), and a step of movement compensation inwhich movement vectors obtained during said movement estimate and notleading to conversion are weighted by a k=m/(t1+t2), 0<k<1 scalar inorder to interpolate at least one Im1 image with m belonging to [t1,t2].
 9. Encoding method as claimed in any of claims 5 to 7, wherein saidsecond encoding implements an estimate of movement after, between anI_(t2) image and an image before I_(t1), and a movement compensationphase in which the movement vectors obtained during said movementestimate and not leading to conversion are weighted by a k′=1−m/(t1+t2),0≦k′≦1 scalar in order to interpolate at least one Im2 image with mbelonging to [t1, t2].
 10. Encoding method as claimed in claims 8 and 9,wherein it implements an after estimate and a before estimate, in orderto obtain an estimated Im image such as Im=aE1+(1-a)E2 with 0≦a≦1. 11.Encoding method as claimed in any of claims 3 to 10, wherein saidselection phase is based on the implementation of an N fixed factorsubsampling, one image in N being encoded using said initial encoding.12. Encoding method as claimed in claim 11, wherein N is greater than 2.13. Encoding method as claimed in either of claims 11 and 12, wherein Nis variable.
 14. Encoding method as claimed in any of claims 3 to 11,wherein said initial encoding implements a transformation on imageblocks and a temporal prediction by blocks.
 15. Encoding method asclaimed in claim 14, wherein said initial encoding is an MPEG-4 or H26Lencoding.
 16. Encoding method as claimed in claim 15, wherein imagesobtained by said MPEG-4 or H26L encoding comprise type I (intra) and/ortype P (predictable) images.
 17. Encoding method as claimed in any ofclaims 3 to 16, wherein said second encoding is based on an M levelhierarchical meshing, M being greater than or equal to
 1. 18. Encodingmethod as claimed in claim 17, wherein said meshing is triangular. 19.Encoding method as claimed in either of claims 17 and 18, wherein itcomprises a management phase of occlusion areas.
 20. Encoding method asclaimed in any of claims 1 to 19, wherein it produces at least two dataflows that can be transmitted over separate transmission paths. 21.Encoding method as claimed in claim 20, wherein said data flows belongto a group comprising: a global header; image data encoded according tosaid initial encoding; image data encoded according to said secondencoding.
 22. Method for decoding an image signal encoded using theencoding method as claimed in any of claims 1 to
 21. 23. Device forencoding an image signal encoded using the encoding method as claimed inany of claims 1 to
 21. 24. Device for decoding an image signal encodedusing the encoding method as claimed in any of claims 1 to
 21. 25.Decoding device as claimed in claim 24, wherein it comprises means fordetermining at least one part of a vector field and/or at least one partof occlusion areas, similar to those implemented during encoding. 26.Storage device with at least one image signal encoded using the encodingprocedure as claimed in any of claims 1 to
 21. 27. System for encoding,transmission and/or decoding an image signal encoded using the encodingprocedure as claimed in any of claims 1 to
 17. 28. System as claimed inclaim 27, wherein a piece of information regarding the choice of one ofsaid encoding modes is known to a decoder according to at least one ofthe techniques that belong to a group comprising the following: apredefined choice, known at the encoding and decoding levels; a piece ofinformation representative of the choice and included in a data flowcomprising at least some encoded image data; a piece of informationrepresentative of the choice and included in a data flow regardlessencoded image data; determining the choice in an intrinsic manner by thedecoder.
 29. Computer program product for encoding and/or decoding imagesignals encoded using the encoding method as claimed in any of claims 1to
 21. 30. Data media hosting a computer program for encoding and/ordecoding an image signal encoded using the encoding method as claimed inany of claims 1 to
 21. 31. Image data signal wherein it comprises dataencoded according to the procedure as claimed in any of claims 1 to 21.32. Signal as claimed in claim 31, wherein it comprises at least oneindicator specifying whether the method as claimed in any of claims 1 to21 is activated or not.
 33. Signal as claimed in either of claims 31 and32, wherein it comprises a data element that specifies the framestructure, at the beginning of a video sequence and/or in each signalframe.
 34. Signal as claimed in any of claims 31 to 33, wherein asequence encoded using said second encoding begins with a headerspecifying a number of frames encoded according to this second encoding.35. Signal as claimed in any of claims 31 to 34, wherein it comprises atleast two data flows that can be transmitted over separate transmissionpaths.
 36. Signal as claimed in claim 35, wherein said data flows belongto a group comprising: a global header; image data encoded according tosaid initial encoding; image data encoded according to said secondencoding.
 37. Application of the method of encoding as claimed in any ofclaims 1 to 21 to at least one of the domains belonging to a groupcomprising: digital TV; real-time video over IP network; real-time videoover network to mobiles; image data storage.